feat(wasm-builder): add support for new `wasm32v1-none` target #7008

StackOverflowExcept1on · 2024-12-27T12:45:45Z

Description

Resolves #5777

Previously wasm-builder used hacks such as -Zbuild-std (required rust-src component) and RUSTC_BOOTSTRAP=1 to build WASM runtime without WASM features: sign-ext, multivalue and reference-types, but since Rust 1.84 (will be stable on 9 January, 2025) the situation has improved as there is new wasm32v1-none target that disables all "post-MVP" WASM features except mutable-globals.

Previously, your rust-toolchain.toml looked like this:

[toolchain]
channel = "stable"
components = ["rust-src"]
targets = ["wasm32-unknown-unknown"]
profile = "default"

It should now be updated to something like this:

[toolchain]
channel = "stable"
targets = ["wasm32v1-none"]
profile = "default"

To build the runtime:

cargo build --package minimal-template-runtime --release

Integration

If you are using Rust 1.84 and above, then install the wasm32v1-none target instead of wasm32-unknown-unknown as shown above. You can also remove the unnecessary rust-src component.

Also note the slight differences in conditional compilation:

wasm32-unknown-unknown: #[cfg(all(target_family = "wasm", target_os = "unknown"))]
wasm32v1-none: #[cfg(all(target_family = "wasm", target_os = "none"))]

Avoid using target_os = "unknown" in #[cfg(...)] or #[cfg_attr(...)] and instead use target_family = "wasm" or target_arch = "wasm32" in the runtime code.

Review Notes

Wasm builder requires the following prerequisites for building the WASM binary:

Rust >= 1.68 and Rust < 1.84:
- wasm32-unknown-unknown target
- rust-src component
Rust >= 1.84:
- wasm32v1-none target
- no more -Zbuild-std and RUSTC_BOOTSTRAP=1 hacks and rust-src component requirements!

github-actions · 2024-12-27T19:24:12Z

Review required! Latest push from author must always be reviewed

prdoc/pr_7008.prdoc

StackOverflowExcept1on · 2025-01-02T13:28:54Z

How do I pass CI checks like SemVer and other?

jasl · 2025-01-08T15:44:56Z

Ping @bkchr It seems the CI won't trigger.

jasl · 2025-01-08T17:06:42Z

@StackOverflowExcept1on https://github.com/paritytech/polkadot-sdk/actions/runs/12674781431/job/35324397805?pr=7008 This required CI failed, could you fix it?

StackOverflowExcept1on · 2025-01-08T17:10:59Z

@jasl @bkchr I'll fix that in the next 30 minutes. Also wondering if we'll have to update the runners tomorrow, since the new version 1.84 will be stable tomorrow. I mean that the wasm32-unknown-unknown target should be replaced by wasm32v1-none on runners. We should come to something one on CI. Perhaps we should also replace all occurrences of “wasm32-unknown-unknown”.

jasl · 2025-01-08T17:39:13Z

@jasl @bkchr I'll fix that in the next 30 minutes. Also wondering if we'll have to update the runners tomorrow, since the new version 1.84 will be stable tomorrow. I mean that the wasm32-unknown-unknown target should be replaced by wasm32v1-none on runners. We should come to something one on CI. Perhaps we should also replace all occurrences of “wasm32-unknown-unknown”.

I'm not Parity staff, so I don't know...
I remember there was a critical accident in history by updating the Rust version, so I propose we not update CI.
Let Parity guys update their facilities.

bkchr · 2025-01-08T18:54:15Z

Actually I realized something, @StackOverflowExcept1on can we change the pr to make the new target optional? So, it should still accept the wasm32-unknown-unknown target, but print some warning.

StackOverflowExcept1on · 2025-01-08T18:57:46Z

@bkchr For now, do you want to use an environment variable like FORCE_NEW_WASM_TARGET to switch to wasm32v1-none and remove the warning?

jasl · 2025-01-08T19:03:45Z

Several required CI checks fail. Maybe the latest patch has trouble

StackOverflowExcept1on · 2025-01-08T19:21:24Z

@jasl I think it's some kind of problem with std hitting wasmv1-none (it doesn't support it). It may have happened after recent merge of master.

jasl · 2025-01-09T01:18:13Z

I believe your original one is good, but I don't know why CI give you the false suggest.

@StackOverflowExcept1on Could you try don't change node = [...] and runtime-full = [...] just change the location of [dependencies.substrate-wasm-builder] ?

jasl · 2025-01-09T14:12:21Z

@bkchr Could you help to re-trigger CI?

@StackOverflowExcept1on I think you still need to apply @bkchr request

Just check for the available targets (I think we already have code for that) and then use the new target if available. If the new target is not available, print a warning to the build output.

Thank you for the work. I'm trying to migrate to Rust 2024 Edition when it's available.

StackOverflowExcept1on · 2025-01-09T14:20:45Z

@jasl I'm still working on this request. It would be nice to have the ability to run CI without pinging the parity team...

No. Just check for the available targets (I think we already have code for that) and then use the new target if available. If the new target is not available, print a warning to the build output.

jasl · 2025-01-09T14:26:32Z

@jasl I'm still working on this request. It would be nice to have the ability to run CI without pinging the parity team...

No. Just check for the available targets (I think we already have code for that) and then use the new target if available. If the new target is not available, print a warning to the build output.

This is a pain point. I guess it is about the cost.
When I fix CI fails, I usually use my local machine to replay failed CI checks, which would help to reduce attempts on GitHub Actions.

If possible, I would propose giving "verified" individual contributors an ability to trigger CI.

bkchr · 2025-01-09T15:43:18Z

These are some annoying security requirements. Which someone apparently turned on again..

jasl · 2025-01-09T15:50:42Z

@bkchr
So https://github.com/paritytech/polkadot-sdk/actions/runs/12687523704/job/35380981212?pr=7008 still gives falsy suggestions...
Do you know who can help this?

StackOverflowExcept1on · 2025-01-09T16:33:10Z

I should probably update prdoc as well: https://github.com/paritytech/polkadot-sdk/actions/runs/12687523719/job/35380940517
The only thing I don't know how to fix:

substrate-wasm-builder (substrate/utils/wasm-builder):
    Change stated in PR Doc: Minor
    Predicted semver change: Patch

prdoc/pr_7008.prdoc

substrate/utils/wasm-builder/src/lib.rs

StackOverflowExcept1on · 2025-01-13T01:01:53Z

@bkchr I think basically did what you wanted without caching. The only problem I have encountered is the need to do a cargo clean.

$ cargo +stable build --package minimal-template-runtime --release
   Compiling minimal-template-runtime v0.0.0 (/home/user/polkadot-sdk/templates/minimal/runtime)
warning: [email protected]: You are building WASM runtime using `wasm32-unknown-unknown` target, although Rust >= 1.84 supports `wasm32v1-none` target!
warning: [email protected]: You can install it with `rustup target add wasm32v1-none --toolchain stable-x86_64-unknown-linux-gnu` if you're using `rustup`.
warning: [email protected]: After installing `wasm32v1-none` target, you must rebuild WASM runtime from scratch, use `cargo clean` before building.
    Finished `release` profile [optimized] target(s) in 3m 28s

StackOverflowExcept1on · 2025-01-16T09:49:19Z

@bkchr @koute friendly ping

substrate/utils/wasm-builder/src/lib.rs

bkchr · 2025-01-17T23:58:35Z

substrate/utils/wasm-builder/src/lib.rs

+	/// Returns whether the `wasm32v1-none` target is installed in this version of the toolchain.
+	fn is_wasm32v1_none_target_installed(&self) -> bool {
+		let dummy_crate = DummyCrate::new(self, RuntimeTarget::Wasm, true);
+		let toolchain = dummy_crate.get_toolchain().expect("toolchain not found");


If get_toolchain returns an error, we should return false.

Also can we please just move get_toolchain() to cargo command?

no, we can't just move get_toolchain() to the cargo command because cargo rustc, which is used by get_toolchain() under the hood, requires --manifest-path for dummy crate:

$ cargo rustc --manifest-path=./app/Cargo.toml -q -- --print sysroot /home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu $ cargo rustc -q -- --print sysroot error: could not find `Cargo.toml` in `/tmp` or any parent directory

If get_toolchain returns an error, we should return false.

ok, I did

let Some(toolchain) = dummy_crate.get_toolchain() else { return false; };

bkchr · 2025-01-18T00:17:48Z

substrate/utils/wasm-builder/src/prerequisites.rs

-				[workspace]
-			"#,
-		);
+		match target {


Why do we need this?

I can't compile fn main() {} for wasm32v1-none because it doesn't have stdlib

so i just decided to compile no_std cdylib with empty panic handler (runtime does the same thing, see wasm_project.rs file)

StackOverflowExcept1on · 2025-01-23T05:58:00Z

@bkchr @koute I've resolved all the conversations. Could you please take a look and run CI?

bkchr · 2025-01-28T22:59:52Z

substrate/utils/wasm-builder/src/lib.rs

+	}
+
+	fn is_target_installed(toolchain: &str, target: &str) -> bool {
+		Command::new("rustup")


https://users.rust-lang.org/t/determining-installed-target-list-when-rustup-is-not-in-use/100594/2

Please use this combined with DummyCrate::().syroot and then check if the folder exists. This way it doesn't require rustup.

feat(wasm-builder): add support for new wasm32v1-none target

78c5f4c

StackOverflowExcept1on requested a review from koute as a code owner December 27, 2024 12:45

fix typo

dcfbeb7

bkchr approved these changes Dec 27, 2024

View reviewed changes

bkchr added the T17-primitives Changes to primitives that are not covered by any other label. label Dec 27, 2024

trying to fix CI

03714a0

github-actions bot requested a review from bkchr December 27, 2024 19:24

Merge branch 'master' into wasm32v1-none

2410530

bkchr approved these changes Dec 30, 2024

View reviewed changes

prdoc/pr_7008.prdoc Show resolved Hide resolved

Update prdoc/pr_7008.prdoc

575d805

koute approved these changes Jan 2, 2025

View reviewed changes

Merge branch 'master' into wasm32v1-none

8fffe18

bkchr enabled auto-merge January 2, 2025 09:55

Merge branch 'master' into wasm32v1-none

ce0fb1d

Merge branch 'master' into wasm32v1-none

3d26ce9

git apply patch from CI

4c3b0ea

auto-merge was automatically disabled January 8, 2025 17:19
Head branch was pushed to by a user without write access

github-actions bot requested review from bkchr and koute January 8, 2025 17:19

bkchr approved these changes Jan 8, 2025

View reviewed changes

just changle location of substrate-wasm-builder

21245bc

Merge branch 'master' into wasm32v1-none

d044efc

Fix umbrella script

1f34087

bkchr reviewed Jan 10, 2025

View reviewed changes

prdoc/pr_7008.prdoc Show resolved Hide resolved

bkchr and others added 2 commits January 10, 2025 12:39

Update prdoc/pr_7008.prdoc

8ae5e7b

add sp-consensus-beefy bump patch to prdoc

f154742

github-actions bot requested a review from bkchr January 10, 2025 17:14

bkchr reviewed Jan 12, 2025

View reviewed changes

substrate/utils/wasm-builder/src/lib.rs Show resolved Hide resolved

StackOverflowExcept1on added 2 commits January 13, 2025 03:57

use wasm32v1-none target only if installed

6bfea51

Merge remote-tracking branch 'origin/master' into wasm32v1-none

5838f1e

github-actions bot requested a review from bkchr January 13, 2025 00:58

StackOverflowExcept1on mentioned this pull request Jan 13, 2025

feat(wasm-builder): switch to new wasm32v1-none target gear-tech/gear#4407

Open

Merge branch 'master' into wasm32v1-none

b21ffdf

bkchr reviewed Jan 18, 2025

View reviewed changes

StackOverflowExcept1on added 2 commits January 18, 2025 07:12

resolve conversations

3628105

Merge remote-tracking branch 'origin/master' into wasm32v1-none

a804037

github-actions bot requested a review from bkchr January 18, 2025 04:13

Merge branch 'master' into wasm32v1-none

d8b1363

Merge branch 'master' into wasm32v1-none

929e755

bkchr reviewed Jan 28, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(wasm-builder): add support for new `wasm32v1-none` target #7008

feat(wasm-builder): add support for new `wasm32v1-none` target #7008

StackOverflowExcept1on commented Dec 27, 2024 •

edited

Loading

github-actions bot commented Dec 27, 2024

StackOverflowExcept1on commented Jan 2, 2025

jasl commented Jan 8, 2025

jasl commented Jan 8, 2025

StackOverflowExcept1on commented Jan 8, 2025 •

edited

Loading

jasl commented Jan 8, 2025

bkchr commented Jan 8, 2025

StackOverflowExcept1on commented Jan 8, 2025

jasl commented Jan 8, 2025

StackOverflowExcept1on commented Jan 8, 2025

jasl commented Jan 9, 2025

jasl commented Jan 9, 2025

StackOverflowExcept1on commented Jan 9, 2025 •

edited

Loading

jasl commented Jan 9, 2025

bkchr commented Jan 9, 2025

jasl commented Jan 9, 2025

StackOverflowExcept1on commented Jan 9, 2025

StackOverflowExcept1on commented Jan 13, 2025

StackOverflowExcept1on commented Jan 16, 2025

bkchr Jan 17, 2025

bkchr Jan 18, 2025

StackOverflowExcept1on Jan 18, 2025

StackOverflowExcept1on Jan 18, 2025

bkchr Jan 18, 2025

StackOverflowExcept1on Jan 18, 2025

StackOverflowExcept1on Jan 18, 2025 •

edited

Loading

StackOverflowExcept1on commented Jan 23, 2025

bkchr Jan 28, 2025

feat(wasm-builder): add support for new wasm32v1-none target #7008

Are you sure you want to change the base?

feat(wasm-builder): add support for new wasm32v1-none target #7008

Conversation

StackOverflowExcept1on commented Dec 27, 2024 • edited Loading

Description

Integration

Review Notes

github-actions bot commented Dec 27, 2024

StackOverflowExcept1on commented Jan 2, 2025

jasl commented Jan 8, 2025

jasl commented Jan 8, 2025

StackOverflowExcept1on commented Jan 8, 2025 • edited Loading

jasl commented Jan 8, 2025

bkchr commented Jan 8, 2025

StackOverflowExcept1on commented Jan 8, 2025

jasl commented Jan 8, 2025

StackOverflowExcept1on commented Jan 8, 2025

jasl commented Jan 9, 2025

jasl commented Jan 9, 2025

StackOverflowExcept1on commented Jan 9, 2025 • edited Loading

jasl commented Jan 9, 2025

bkchr commented Jan 9, 2025

jasl commented Jan 9, 2025

StackOverflowExcept1on commented Jan 9, 2025

StackOverflowExcept1on commented Jan 13, 2025

StackOverflowExcept1on commented Jan 16, 2025

bkchr Jan 17, 2025

Choose a reason for hiding this comment

bkchr Jan 18, 2025

Choose a reason for hiding this comment

StackOverflowExcept1on Jan 18, 2025

Choose a reason for hiding this comment

StackOverflowExcept1on Jan 18, 2025

Choose a reason for hiding this comment

bkchr Jan 18, 2025

Choose a reason for hiding this comment

StackOverflowExcept1on Jan 18, 2025

Choose a reason for hiding this comment

StackOverflowExcept1on Jan 18, 2025 • edited Loading

Choose a reason for hiding this comment

StackOverflowExcept1on commented Jan 23, 2025

bkchr Jan 28, 2025

Choose a reason for hiding this comment

feat(wasm-builder): add support for new `wasm32v1-none` target #7008

feat(wasm-builder): add support for new `wasm32v1-none` target #7008

StackOverflowExcept1on commented Dec 27, 2024 •

edited

Loading

StackOverflowExcept1on commented Jan 8, 2025 •

edited

Loading

StackOverflowExcept1on commented Jan 9, 2025 •

edited

Loading

StackOverflowExcept1on Jan 18, 2025 •

edited

Loading